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CLAIMS: Please amend the claims according to the status designations in the following list, 
which contains all claims that were ever in the application, with the text of all active claims. 
1.-21. (CANCELED) 

22. (NEW) A method for enhancing facial images of a person in a video by superimposing 
virtual object images onto said facial images automatically and dynamically in real-time, 
comprising the following steps of: 

(a) capturing a plurality of images for said person with a single or a plurality of means for 
capturing images, 

(b) processing said plurality of images in order to detect the face of said person in said plurality 
of images for an initial face detection, 

(c) tracking the detected face in real-time, 

(d) verifying said face within the boundary of the face tracking window using a second local face 
detection, 

(e) calculating facial feature coordinate information from said face by applying a plurality of 
facial feature detection approaches and applying a block-processing, 

(f) processing a touch-free interaction by said person to select virtual object images, and 

(g) superimposing said virtual object images onto said facial images automatically and 
dynamically in real-time, 

wherein exemplary facial images comprise eyes, nose, and mouth on the face, 

wherein exemplary virtual object images comprise a pair of sunglasses image, hat image, or any 

other predefined virtual objects, which can be attached to and enhance the facial images, 
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whereby the verification of said face within the boundary of the face tracking window makes 
more accurate facial images to be passed on to the next step while enabling real-time tracking of 
the face, and 

whereby said touch-free interaction enables said person to choose said virtual object images on a 
means for displaying output to superimpose onto said facial images without touching any device. 

23. (NEW) The method according to claim 22, wherein the method further comprises a step of 
repeating the steps from (a) to (g) for a plurality of persons. 

24. (NEW) The method according to claim 22, wherein the method further comprises a step of 
estimating regions of interest for each facial feature in said face dynamically, 

whereby said regions of interest change according to the results from the step of verifying said 
face within the boundary of the face tracking window, and 

whereby said regions of interest are used as boundaries for detecting each facial feature, such as 
eyes, nose, and mouth on said face. 

25. (NEW) The method according to claim 22, wherein the method further comprises a step of 
applying fusion algorithms and geometrical constraints to said facial feature coordinate 
information. 

26. (NEW) The method according to claim 22, wherein the method further comprises a step of 
smoothing said facial feature coordinate information. 
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27. (NEW) The method according to claim 26, wherein the method further comprises a step of 
storing a history of a plurality of said facial feature coordinate information and applying a 
smoothing algorithm for the current facial feature coordinate using said history of said facial 
feature coordinate information. 

28. (NEW) The method according to claim 22, wherein the method further comprises a step of 
combining the face detection process and the real-time face tracking process, 

whereby the integration enables efficient and robust real-time facial image processing. 

29. (NEW) The method according to claim 22, wherein the method further comprises a step of 
applying said block-processing to each of said plurality of facial feature detection approaches. 

30. (NEW) The method according to claim 22, wherein the step of (g) superimposing said virtual 
object images onto said facial images automatically and dynamically in real-time further 
comprises steps of: 

(a) preparing virtual object images, 

(b) validating said facial feature coordinate information, 

(c) smoothing, 

(d) aesthetic processing, and 

(e) processing final superimposition. 
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31. (NEW) The method according to claim 30 5 wherein the step of (a) preparing virtual object 
images further comprises a step for setting pivot points in said virtual object images. 

32. (NEW) The method according to claim 30, wherein the step of (d) aesthetic processing 
further comprises a step for processing rotation and translation of said virtual object images. 

33. (NEW) An apparatus for enhancing facial images of a person in a video by superimposing 
virtual object images onto said facial images automatically and dynamically in real-time, 
comprising: 

(a) means for capturing a plurality of images for said person with a single or a plurality of means 
for capturing images, 

(b) means for processing said plurality of images in order to detect the face of said person in said 
plurality of images for an initial face detection, 

(c) means for tracking the detected face in real-time, 

(d) means for verifying said face within the boundary of the face tracking window using a second 
local face detection, 

(e) means for calculating facial feature coordinate information from said face by applying a 
plurality of facial feature detection approaches and applying a block-processing, 

(f) means for processing a touch-free interaction by said person to select virtual object images, 
and 

(g) means for superimposing said virtual object images onto said facial images automatically and 
dynamically in real-time, 
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wherein exemplary facial images comprise eyes, nose, and mouth on the face, 
wherein exemplary virtual object images comprise a pair of sunglasses image, hat image, or any 
other predefined virtual objects, which can be attached to and enhance the facial images, 
whereby the verification of said face within the boundary of the face tracking window makes 
more accurate facial images to be passed on to the next means while enabling real-time tracking 
of the face, and 

whereby said touch-free interaction enables said person to choose said virtual object images on a 
means for displaying output to superimpose onto said facial images without touching any device. 

34. (NEW) The apparatus according to claim 33, wherein the apparatus further comprises means 
for repeatedly using the means from (a) to (g) for a plurality of persons. 

35. (NEW) The apparatus according to claim 33, wherein the apparatus further comprises means 
for estimating regions of interest for each facial feature in said face dynamically, 

whereby said regions of interest change according to the results from using the means for 
verifying said face within the boundary of the face tracking window, and 
whereby said regions of interest are used as boundaries for detecting each facial feature, such as 
eyes, nose, and mouth on said face. 

36. (NEW) The apparatus according to claim 33, wherein the apparatus further comprises means 
for applying fusion algorithms and geometrical constraints to said facial feature coordinate 
information. 
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37. (NEW) The apparatus according to claim 33, wherein the apparatus further comprises means 
for smoothing said facial feature coordinate information. 

38. (NEW) The apparatus according to claim 37, wherein the apparatus further comprises means 
for storing a history of a plurality of said facial feature coordinate information and applying a 
smoothing algorithm for the current facial feature coordinate using said history of said facial 
feature coordinate information. 

39. (NEW) The apparatus according to claim 33, wherein the apparatus further comprises means 
for combining the face detection process and the real-time face tracking process, 

whereby the integration enables efficient and robust real-time facial image processing. 

40. (NEW) The apparatus according to claim 33, wherein the apparatus further comprises means 
for applying said block-processing to each of said plurality of facial feature detection approaches. 

41. (NEW) The apparatus according to claim 33, wherein the means for (g) superimposing said 
virtual object images onto said facial images automatically and dynamically in real-time further 
comprises means for: 

(a) preparing virtual object images, 

(b) validating said facial feature coordinate information, 

(c) smoothing, 
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(d) aesthetic processing, and 

(e) processing final superimposition. 

42. (NEW) The apparatus according to claim 41, wherein the means for (a) preparing virtual 
object images further comprises means for setting pivot points in said virtual object images. 

43. (NEW) The apparatus according to claim 41, wherein the means for (d) aesthetic processing 
further comprises means for processing rotation and translation of said virtual object images. 



